Publication Date: 6/1/74 Pages: 2 Date Entered: 2/22/84 Title: RECOMMENDED PRACTICE FOR DEALING WITH OUTLYING OBSERVATIONS June 1974 U.S. ATOMIC ENERGY COMMISSION REGULATORY GUIDE DIRECTORATE OF REGULATORY STANDARDS REGULATORY GUIDE 5.36 RECOMMENDED PRACTICE FOR DEALING WITH OUTLYING OBSERVATIONS A. INTRODUCTION Section 70.51, "Material Balance, Inventory, and Records Requirements," of 10 CFR Part 70, "Special Nuclear Material," requires certain licensees authorized to possess special nuclear material to establish and maintain sufficient written material control and accounting procedures to enable the licensee to account for the special nuclear material in his possession. Section 70.22, "Contents of Applications," requires applicants for certain AEC licenses for special nuclear material to submit to the Commission as part of the application a full description of such procedures, including recording, reporting, physical inventory, and measurement procedures for special nuclear material. Essential to satisfactory material control and accounting procedures are measurement and statistical control. This guide identifies methods and procedures acceptable to the Regulatory staff for dealing with the problem of outlying observations in samples and for testing their statistical significance. B. DISCUSSION Committee E-11 on Statistical Methods of the American Society for Testing and Materials (ASTM) has revised a standard that deals with the problem of outlying observations in samples and how to test their statistical significance. An outlying observation, or "outlier," is one that appears to deviate markedly from other members of the sample in which it occurs. In this connection, the following two alternatives are of interest: 1. An outlying observation may be merely an extreme manifestation of the random variability inherent in the data. If this is the case, the value should be retained and processed in the same manner as the other observations in the sample. 2. On the other hand, an outlying observation may be the result of gross deviation from prescribed experimental procedure or an error in calculating or recording the numerical value. In such cases, it may be desirable to institute an investigation to ascertain the reason for the aberrant value. The observation may even eventually be rejected as a result of the investigation, though not necessarily so. At any rate, in subsequent data analysis, the outlier or outliers will be recognized as probably being from a different process than that of the sample values. The committee's purpose in developing this revised standard is to provide statistical rules that will lead the experimenter almost unerringly to look for causes of outliers when they really exist, and hence to decide whether alternative 1 above is the more plausible hypothesis to accept, as compared with alternative 2, in order that the most appropriate action in further data analysis may be taken. The procedures covered therein apply primarily to the simplest kind of experimental data, that is, replicate measurement of some property of a given material or observations in a supposedly single random sample. Nevertheless, the tests suggested do cover a wide enough range of cases in practice to have broad utility. Topics addressed in this standard are: a. Basis of statistical criteria for outliers b. A single observation in a sample c. The least and the greatest observation in a sample d. The two largest or the two smallest observations as probable outliers e. Rejection of several outliers f. Recommended criterion using independent standard deviation g. Recommended criteria for known standard deviation The revised standard was approved by ASTM Committee E-11 in February 1974 and will be published in August 1974 with the designation E178-74.(1)C. REGULATORY POSITION The recommended practices dealing with the problem of outlying observations in samples and the methods for testing their statistical significance contained in ASTM Standard E178-74,(1) "Recommended Practice for Dealing with Outlying Observations," are generally acceptable and provide an adequate basis for determining if an outlying observation is an aberrant value or an extreme manifestation of the random variability inherent in the data, subject to the following qualifications: ---------- (1) Copies may be obtained from the American Society for Testing and Materials, 1916 Race Street, Philadelphia, Pennsylvania 19103. ---------- 1. In regard to the statistical criteria discussed in Section 3.1 of the ASTM Standard, the level of significance should not exceed 0.5 percent. 2. Statistical outlier tests should be applied to basic measurement control processes, but not directly to the rejection of observed MUF values. 3. Caution should be applied to avoid unwarranted rejection of suspect observations. Discarding an overly large number of false outliers could introduce bias rather than eliminate it. 4. In regard to Section 3.2, the assumption of normality should be assessed using Regulatory Guide 5.22, "Assessment of the Assumption of Normality (Employing Individual Observed Values)," April 1974. 5. All discarded outliers should be recorded as part of the records maintained for special nuclear material control purposes. 32